Modular scalable switch architecture

ABSTRACT

A scalable Ethernet switch includes modules which can be interconnected to provide a single, virtual switch. The modules may be of uniform physical size and shape relative to a standard telecommunications rack. When greater capacity is required, an additional module is mounted in a rack and interconnected with the other modules, e.g., in a bi-directional ring. Enhanced port density is provided by interconnecting the modules with 12 GbE links which operate on standard 10 GbE wiring and connectors. Consequently, throughput between modules in increased relative to standard 10 GbE links without increasing form factor. Further, transmission power control can be implemented such that modules of the virtual switch may be physically adjacent or separated by distances of several meters.

FIELD OF THE INVENTION

This invention relates generally to the field of network communications,and more particularly to scalable switches.

BACKGROUND OF THE INVENTION

Historically, service providers have expended considerable effort toestimate equipment requirements, including how those requirements willchange over time, in order to minimize cost while accommodating demand.Accurate estimation was important because over-estimating requirementscould lead to over-provisioning with costly equipment. If, for example,a service provider estimates that for a given location there will be arequirement for equipment capable of throughput X for the first fewyears following introduction of a service, and that equipment capable ofthroughput 100X may be required within ten years, then it may not beeconomically rational to install the equipment capable of throughput100X at service introduction. Rather, the service provider may wish toincrease capacity gradually in relation to increased demand. However,there are practical limitations to gradually increasing capacity. First,each installation of new equipment has an overhead cost apart from thatof the equipment itself, so it is desirable to reduce the frequency ofnew installations. Second, if increasing capacity requires that lowercapacity equipment is replaced with higher capacity equipment then eachcapacity increase may produce excess equipment for which there may be noprofitable use.

Scalable equipment mitigates some of the problems described above byfacilitating gradual increases in capacity. One technique for creating ascalable switch is to use multiple I/O line cards interconnected via abackplane and switch fabric. The switch can then be scaled-up by addingline cards. Advantages of this technique include ease of installationand not producing excess equipment. However, the chassis which housesthe fabric and line cards is typically of a size capable ofaccommodating a full complement of line cards. Hence, there is sizeinefficiency in a partially empty chassis. Such size inefficiency is aproblem where space is costly. Size inefficiency is particularlyproblematic in markets where competing service providers co-locateequipment, and relatively newer providers rent space from incumbentservice providers. It would therefore be desirable to have equipmentthat could scale gradually in physical size and capacity withoutproducing excess equipment and without requiring complex installation.

SUMMARY OF THE INVENTION

In accordance with one embodiment of the invention, a scalable networkdevice comprises: a plurality of switch modules, each switch modulehaving at least one internal port and at least one external port,wherein the external ports operate at a standard line rate and theinternal ports operate at a non-standard line rate, the non-standardline rate internal ports being implemented with connectors rated at aline rate lower than the non-standard line rate.

In accordance with another embodiment of the invention, a method forscaling capacity of a network device comprises the steps of: adding, tothe device, additional switching modules, each switching module havingat least one internal port and at least one external port, wherein theexternal ports operate at a standard line rate and the internal portsoperate at a non-standard line rate, the non-standard line rate internalports being implemented with connectors rated at a line rate lower thanthe non-standard line rate.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a diagram of a scalable, non-blocking switch architectureincluding modules interconnected in a ring.

FIG. 2 is a block diagram of a switch module of FIG. 1.

FIG. 3 illustrates the interconnections between the switch modules andthe network in greater detail.

DETAILED DESCRIPTION

Referring to FIGS. 1, 2 and 3, a scalable, non-blocking switch includesat least one module (100 a) which is both independently andinterdependently functional as a switch. The switch is scaled by addingmore modules (100 b, 100 c, 100 d). In the illustrated embodiment, up totwelve modules may be combined in a single switch. Each module is housedin a separate chassis (102) having a form factor designed to efficientlyuse the standard amount of space available in a telecommunications rack(104).

In the illustrated embodiment, when multiple modules (100 a-d) areemployed to provide a single, scaled-up switch, the individual modulesare interconnected in a ring architecture. The ring includes twobidirectional logical links (300 a, 300 b) each flowing in opposingdirections around the overall interconnect ring, thereby providingprotection from link or node failure. For example, if switch (100 a)fails then traffic bound for switch (100 d) from switch (100 b) canreach switch (100 d) by being redirected via switch (100 c).

Each module (100 a-d) includes a plurality of external ports (106)operable to interconnect with other network devices, i.e., networkdevices other than the modules of the switch. These external portsemploy standard connectors, such as RJ45 connectors, and supportstandard link line rates such as 1 gigabit (“Gb”) per second and 10Gbps. Typically, subsets of the entire set of external ports will beused to provide different line rates.

Internal ports (108) are available for interconnection with othermodules of the switch. These ports are “internal” in the sense that theyare dedicated to communications with other modules of the switch. Unlikethe external ports, at least some of the internal ports supportnon-standard Ethernet links which operate at a line rate. of 12 Gb. The12 GbE internal links utilize standard 12× Infiniband™ connectors and12× twin-axial cabling. Hence, the form factor relative to 10 GbE linksis unchanged, although throughput is increased. This increase inthroughput without a corresponding increase in form factor permits theswitch to scale better. In particular, more modules may be added withoutunacceptable degradation in performance relative to 10 GbE links.Further, the use of 12 Gb links simplifies implementation where thechipset has native 12 Gb links for chip-to-chip communication. Stillfurther, the 12 Gb links may be backward compatible with 10 Gb linkssupported by legacy equipment. The illustrated embodiment has one 12×Infiniband™ connector for each direction in the ring. The 12×Infiniband™ connectors may have a rated capacity of 10 Gb but be capableof 12 Gb communication at the relatively short distances between switchmodules.

Programmable power control enables flexible installation of the switchmodules (100 a-d). Unlike switches which scale by adding line cards, forwhich distances between card slots are known in advance, it is notgenerally known in advance what rack space will be available when addinga switch module. It may be possible to mount modules adjacent to oneanother in some circumstances, and it may be necessary to mount a moduleseveral meters away from other modules in other circumstances. In orderto accommodate differences in distance (and length of cabling) betweenmodules, each module is equipped with programmable, per-porttransmission power control. For example, an Application SpecificIntegrated Circuit (ASIC) (200) may be operable in response to inputfrom a field technician of the length of cabling to index into a tableto obtain a transmission power level.

While the invention is described through the above exemplaryembodiments, it will be understood by those of ordinary skill in the artthat modification to and variation of the illustrated embodiments may bemade without departing from the inventive concepts herein disclosed.Moreover, while the preferred embodiments are described in connectionwith various illustrative structures, one skilled in the art willrecognize that the system may be embodied using a variety of specificstructures. Accordingly, the invention should not be viewed as limitedexcept by the scope and spirit of the appended claims.

1. A scalable network device comprising: a plurality of switch modules,each switch module having at least one internal port and at least oneexternal port, wherein the external ports operate at a standard linerate and the internal ports operate at a non-standard line rate, thenon-standard line rate internal ports being implemented with connectorsrated at a line rate lower than the non-standard line rate such thatconnector rating is exceeded during operation, and wherein the internalports are employed for communication between switch modules and theexternal ports are employed for communication with devices other thanswitch modules of the switch.
 2. The device of claim 1 wherein theswitch modules are interconnected in a ring having at least first andsecond logical links that operate in opposing direction.
 3. The deviceof claim 1 wherein each module independently switches at least sometraffic.
 4. The device of claim 1 wherein each of the switch modules hasa chassis, and the chassis each has the same dimensions.
 5. The deviceof claim 1 wherein the internal ports are backward compatible with alesser standard line rate.
 6. The device of claim 1 wherein the lengthof cabling between switch modules differs between module pairs.
 7. Thedevice of claim 6 wherein each module includes at least one chip thatoperates to set transmission power level on at least one internal port.8. The device of claim 7 wherein the chip operates in response to anindication of cabling length to select a transmission power level.
 9. Amethod for scaling capacity of a network device comprising the steps of:adding, to the device, additional switching modules, each switchingmodule having at least one internal port and at least one external port,wherein the external ports operate at a standard line rate and theinternal ports operate at a non-standard line rate, the non-standardline rate internal ports being implemented with connectors rated at aline rate lower than the non-standard line rate such that connectorrating is exceeded during operation, and wherein the internal ports areemployed for communication between switching modules and the externalports are employed for communication with devices other than switchingmodules of the switch.
 10. The method of claim 9 including the furtherstep of interconnecting the switching modules in a ring having at leastfirst and second logical links that operate in opposing direction. 11.The method of claim 9 wherein each switching module independentlyswitches at least some traffic.
 12. The method of claim 9 wherein eachof the switch modules has a chassis, and the chassis each has the samedimensions.
 13. The method of claim 9 wherein the internal ports arebackward-compatible with a lesser standard line rate.
 14. The method ofclaim 9 wherein the length of cabling between switching modules differsbetween module pairs.
 15. The method of claim 14 including the furtherstep of setting transmission power level on at least one internal port.16. The method of claim 15 including the further step of selecting atransmission power level in response to an indication of cabling length.